System and method for associating presented digital content within recorded digital stream and method for its playback from precise location

ABSTRACT

To avoid discontinuities when resuming play of a digital stream that was paused while being viewed live and being simultaneously recorded, the timestamp and digital stream time base discontinuity count associated with the last-viewed frame are used to access the frame in storage that has the same discontinuity count (indicating that it is in the same segment as the last-viewed frame) and a timestamp one greater than the timestamp of the last-viewed frame.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.10/830,996, filed Apr. 22, 2004, which claims priority from U.S.provisional patent application Ser. No. 60/539,297, filed Jan. 26, 2004.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to television systems.

2. Description of the Related Art

A user of a personal video recorder (PVR) conveniently may record avideo stream at the same time the user is viewing the stream live, i.e.,as the stream is being received by the user's television system. If theuser desires to leave the room but not miss any part of the programbeing recorded, the user may manipulate a remote control device to causethe system to “pause”, i.e., to freeze a frame on the TV display whilecontinuing to record the program.

When the user wants to resume viewing the program, he again manipulatesthe remote control device to cause play to resume (e.g., by pressing a“play” button on the remote device). However, current solutions toproviding for a smooth transition in the video from “pause” to resumedplay have drawbacks, which can be better understood after the followingbrief discussion.

Digital multimedia streams may be sent to receivers in a format known asMoving Pictures Expert Group (MPEG) standards such as MPEG-1, MPEG-2(also used for DVD format), MPEG-4 and other block based transformcodecs. In MPEG formatting, the data is encoded using MPEG principles,sent to the receiver, and then decoded at the receiver.

As recognized herein, MPEG frames may have timestamps (sometimesreferred to as presentation timestamps, or PTS for short) that can beused to identify the frames. As also understood herein, however, a PTSmay not be unique, because MPEG streams can undergo time basediscontinuities between frame segments, after which PTS numbers canrepeat. Accordingly, while a PTS value identifies a frame within asegment, it does not necessarily uniquely identify the frame within theentire stream.

With the above in mind, one way to resume play is simply not to careabout the possibility of discontinuity and either resume playing thelive feed or resume playing using the recorded feed using, as a bestguess of where the video was paused, the PTS of the last live frame thatwas displayed at the time of pause. For reasons set forth above this canresult in undesired discontinuities that hinder viewing enjoyment.

A second way to resume play is to essentially never show the live feed,but rather to always display the video from the recorded stream asthough it were the live feed. While this avoids discontinuities becausethe live feed is never used for display, it results in latency (delays)particularly if the user changes channels. With the above drawbacks inmind, the solutions herein have been provided.

SUMMARY OF THE INVENTION

The invention provides systems and methods for precisely associating andlocating viewed digital content such as video frame with stored digitalcontent using a pair of key information, namely, discontinuity countsand timestamps. The invention permits a user to precisely access anypreferred video frame. Particular uses of the invention are in pausingand resuming play of a video stream, as well as bookmark operations forlater access for viewing or editing. The digital data may be video oraudio or other data whose presentation is based on timestamps.

In one aspect the invention precisely associates and locates vieweddigital content such as video frames with stored digital content byusing a pair of key information, namely, discontinuity count andtimestamp (e.g., PTS) information. With the invention disclosed herein,a user may access digital content in precise locations, such as aspecified video frame. In non-limiting examples the invention is usefulfor precise video pause/resume operations, bookmark applicationsincluding jumping to a certain video frame which is stored in a storagemedium, and video stream editing applications. The invention is usefulnot only with video data but also audio data and other data, and it maybe used to associate any presented digital content with stored data.

Accordingly, a television system includes a processor which associatesat least one timestamp of at least one digital information element, suchas a video frame, with at least one discontinuity count. A displaypresents digital data such as video streams from the processor, and astorage stores digital data such as video streams. The timestamp anddiscontinuity count are used by the processor for resuming play, usingdata in the storage, of a digital data stream such as a video streamthat was paused or bookmarked during live play of the stream. Theprocessor may be associated with a personal video recorder.

The preferred processor uses the timestamp and discontinuity count of alast-presented frame in a live digital stream to resume play usingdigital data in the storage. Digital frames in the storage areassociated with respective timestamps and digital stream discontinuitycounts.

In another aspect, a method for providing for smooth resumption of playof a digital stream includes sending the stream to a decoder for livedisplay of the stream on a display. Also, the method includes sendingthe stream to a storage for storage of the stream, with frames of thestream that are sent to the storage being associated with respectivetimestamps and digital stream discontinuity counts. A “pause” commandcan be received, in which case the timestamp and discontinuity count ofthe paused frame are recorded. Upon receipt of a “resume play” command,using the timestamp and discontinuity count of the paused frame a frameis retrieved from the storage based on the frame having a discontinuitycount matching the discontinuity count of the paused frame, andpreferably based on the frame having the next sequential timestamp.

In still another aspect, a television system includes a TV display, adecoder driving the display, and means for sending a received digitalstream having time base discontinuities through the decoder forpresentation thereof on the display. Means are provided for pausing thedisplay. Also, the system includes means for sending a recorded versionof the received digital stream through the decoder for presentationthereof on the display upon resumption of play, with the means forsending a recorded version accounting for time base discontinuities inthe stream.

BRIEF DESCRIPTION OF THE DRAWINGS

The details of the present invention, both as to its structure andoperation, can best be understood in reference to the accompanyingdrawings, in which like reference numerals refer to like parts, and inwhich:

FIG. 1 is a block diagram of the system of the present invention;

FIG. 2 is a flow chart of the present logic;

FIG. 3 is a schematic diagram illustrating the display process using thelive video feed prior to entering a “pause” command, showing bothlogical and hardware system blocks and omitting some components forclarity that are not essential for understanding FIG. 3; and

FIG. 4 is a schematic diagram of the display process using recordedvideo feed after entering a “pause” command followed by a “resume”command, showing both logical and hardware system blocks and omittingsome components for clarity that are not essential for understandingFIG. 4.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring initially to FIG. 1, a television (TV) system is shown,generally designated 10, that includes an audio/video TV display 12 thatconventionally receives, potentially through a processor or processors14 that may be housed in a set-top box or personal video recorder (PVR),televised content from an antenna, satellite dish, cable, etc. fordisplay of the content. The processor 14 alternatively can beincorporated into the housing of the display 12 to function inaccordance with the disclosure herein, or it can be implemented byplural processors (e.g., one in a PVR and one in the TV or set-top box)acting in concert with each other. As shown in FIG. 1, the content fromthe processor 14 is decoded by a decoder 16 in accordance withprinciples known in the art prior to display. Also, content being viewedlive on the display 12 may be recorded and stored on a storage 18, e.g.,an optical or magnetic disk or other memory media that can be part of aPVR.

It is to be understood that the term “television” or “television system”encompasses any apparatus that has a television tuner and thebelow-described capability in a single housing or in separate housingsthat cooperate together. For instance, the processor 14 alternativelycan be incorporated into a personal video recorder (PVR) that functionsin accordance with the present invention, or even into a standalonecomputer such as a PC or laptop with its own monitor (not shown), andcan communicate with the display 12 by wired or wireless link or simplyby transferring data from the TV to the computer.

In the preferred non-limiting embodiment shown, the processor 14 mayaccess one or more software or hardware elements to undertake thepresent logic.

Additionally, the processor 14 can be associated with a receiver forconventionally receiving control signals from a portable remote controldevice 20 that functions in accordance with principles known in the art.It is to be understood that while FIG. 1 shows that the device 20 can bea conventional TV remote control device, less preferably other devicescan be used, such as but not limited to keyboards, keypads, mice, touchscreen technology, voice activation/recognition technology, etc.Conventional channel, volume, and TV settings buttons (not shown) canalso be provided. Using the remote 20, a user can, e.g., play a stream,record a stream, pause a stream, bookmark a location in a stream, etc.

The processor 14 executes the logic set forth herein. The flow chartsherein illustrate the structure of the logic modules of the presentinvention as embodied in computer program software. Those skilled in theart will appreciate that the flow charts illustrate the structures oflogic elements, such as computer program code elements or electroniclogic circuits, that function according to this invention. Manifestly,the invention is practiced in its essential embodiment by a machinecomponent that renders the logic elements in a form that instructs adigital processing apparatus (that is, a computer or microprocessor) toperform a sequence of function steps corresponding to those shown.Internal logic could be as simple as a state machine.

In other words, the present logic may be established as a computerprogram that is executed by a processor within, e.g., the presentmicroprocessors/servers as a series of computer-executable instructions.In addition to residing on hard disk drives, these instructions mayreside, for example, in RAM of the appropriate computer, or theinstructions may be stored on magnetic tape, electronic read-onlymemory, or other appropriate data storage device.

The logic of the present invention can be seen in reference to FIG. 2.Commencing at block 22, a video stream feed is received live from asource of video, e.g., from a cable or satellite transmitter head end orwirelessly from a broadcaster. At block 24, the live feed of the videostream is sent through the decoder 16 for display on the display 12.Also, the video stream is recorded by storing it in the storage 18.

Block 26 indicates that as the stream is stored, both the timestamps(e.g., PTS) of the frames and a count of discontinuities associated withthe video segment of each frame are recorded or otherwise associatedwith the stored frames. More specifically, a counter that may beimplemented by the processor 14 is initialized at zero at the start ofreceiving the stream to count up by ones for each time basediscontinuity it detects in the stream. The counter value at the time aframe is recorded is associated with that frame, along with itstimestamp, which may be obtained from the frame header.

In a preferred implementation counters are provided in the live feed(display) line and the recorded video line, so that the discontinuitycount of the frame from the live stream being displayed is known, as isthe count for all frames being stored in the storage 18. In the eventthat frames come unaccompanied by timestamps, timestamps may becalculated for the frames using a clock accessible by the processor 14.This preferred timestamp compensation guarantees that all the recordedframes have respective timestamp entries.

At block 28, a command to pause the video display may be received when,for example, a user appropriately manipulates the remote control device20. The timestamp of the last-displayed frame along with its time basediscontinuity count are recorded.

At block 30, a command to resume play is received, and the logic thenmoves to block 32 to look up the next frame at which video presentationwill be resumed. To do this, the logic matches the discontinuity countof the last-displayed frame from the live video stream feed to thediscontinuity count of a segment of frames in the storage 18, and thenselects either the frame from the count-matching segment that has thesame timestamp as the last-displayed frame or more preferably that hasthe next sequentially higher timestamp. Less preferably, frames havingother timestamps close to the two mentioned above may be selected withinthe same segment.

Starting with the frame selected at block 32, the video stream stored inthe storage 18 is sent to the decoder 16 at block 34 for resumingpresentation of the video stream on the display 12 using the data thatis recorded in the storage 18, without any discontinuities.

The above process is illustrated in FIGS. 3 and 4. Reference numeral 36in FIG. 3 represents the video stream that is received from the videosource (referred to herein as the “live feed” or “live data”). Atlogical location 38 the live feed is sent both to the storage 18 along arecording path and to the decoder 16 for display along a decoding path.As the video stream is being recorded, blocks 40 and 42 in FIG. 3 showthat a discontinuity counter in the recording path counts time basediscontinuities in the stream (by observing such discontinuities in thestream in accordance with MPEG principles known in the art) while atimestamp reporter notes the timestamps in the headers of the frames ofthe stream. Also, as indicated at logical state 43, the video stream isstored frame by frame, with each frame being associated with itsrespective timestamp and with the count that the discontinuity counter42 had when the particular frame was received.

Also, as shown in FIG. 3 a decoding path discontinuity counter 44 andtimestamp reporter 46 note the timestamps and discontinuity count as thelive feed is displayed. Note that in both FIG. 3 and FIG. 4 scenarios,the discontinuity counters 42, 44 at both the decode path and recordpath are reset to the same value when recording is commenced toguarantee discontinuity counting is synchronized between both paths.

As indicated at logical state 48, the data being displayed may bebuffered in the decoder 16 if desired. When a “pause command” (or abookmark command) is received, as indicated at 50 in FIG. 3 thetimestamp and discontinuity count of the last-displayed frame (referredto for illustration as a frame having a timestamp of “t0” anddiscontinuity count of “n”) is recorded.

FIG. 4 shows the same components as FIG. 3 after a command has beenreceived to pause and then to resume play. As indicated at 50 in FIG. 4,the last-displayed frame (timestamp=t0, discontinuity count=n) is in thestorage 18, as is the next successive video frame (i.e., timestamp=t1,discontinuity count=n). In the preferred embodiment play is resumed bysending, to the decoder 16 from storage 18, the next successive frame(timestamp=t1) in the same video segment as the last-displayed frame(indicated by having a matching discontinuity count=n).

While the particular SYSTEM AND METHOD FOR ASSOCIATING PRESENTED DIGITALCONTENT WITHIN RECORDED DIGITAL STREAM AND METHOD FOR ITS PLAYBACK FROMPRECISE LOCATION as herein shown and described in detail is fullycapable of attaining the above-described objects of the invention, it isto be understood that it is the presently preferred embodiment of thepresent invention and is thus representative of the subject matter whichis broadly contemplated by the present invention, that the scope of thepresent invention fully encompasses other embodiments which may becomeobvious to those skilled in the art, and that the scope of the presentinvention is accordingly to be limited by nothing other than theappended claims, in which reference to an element in the singular means“at least one”. All structural and functional equivalents to theelements of the above-described preferred embodiment that are known orlater come to be known to those of ordinary skill in the art areexpressly incorporated herein by reference and are intended to beencompassed by the present claims. Moreover, it is not necessary for adevice or method to address each and every problem sought to be solvedby the present invention, for it to be encompassed by the presentclaims. Furthermore, no element, component, or method step in thepresent disclosure is intended to be dedicated to the public regardlessof whether the element, component, or method step is explicitly recitedin the claims. No claim element herein is to be construed under theprovisions of 35 U.S.C. §112, sixth paragraph, unless the element isexpressly recited using the phrase “means for”.

1. A television systems comprising: at least one processor associatingat least one timestamp of at least one digital data element with atleast one discontinuity count; wherein the timestamp and discontinuitycount are used by the processor for resuming play, using data in astorage accessible to the processor, of a digital data stream that waspaused and/or bookmarked during live play of the stream.
 2. The systemof claim 1, wherein the processor is associated with a personal videorecorder.
 3. The system of claim 1, wherein the digital data elementincludes at least one video frame, and the processor uses the timestampand discontinuity count of a last-presented video frame in a live videostream to resume play using video data in a storage.
 4. The system ofclaim 3, wherein video frames in the storage are associated withrespective timestamps and video stream discontinuity counts.
 5. Thesystem of claim 4, wherein if a frame comes unaccompanied by anassociated timestamp, a timestamp is calculated for the frame.
 6. Thesystem of claim 4, comprising a record discontinuity counter in arecording path and a decode discontinuity counter in a decoding path,and the counters are synchronized with each other.
 7. A method forproviding for smooth resumption of play of a digital stream, comprising:detecting a “resume play” command; using a timestamp and discontinuitycount of a paused and/or bookmarked frame, retrieving a frame at leastpartially based on the frame having a discontinuity count matching thediscontinuity count of the paused and/or bookmarked frame; andsynchronizing a discontinuity count in a recording path with adiscontinuity count in a decoding path.
 8. The method of claim 7,further comprising retrieving a frame based on the frame having atimestamp that matches the timestamp of the paused frame.
 9. The methodof claim 7, further comprising retrieving a frame based on the framehaving a timestamp that is the next sequential timestamp after thetimestamp of the paused frame.
 10. The method of claim 7, comprisingusing a personal video recorder to execute at least some of the methodacts.
 11. (canceled)
 12. The method of claim 7, comprising generating atimestamp for a frame arriving without a timestamp.
 13. A televisionsystem, comprising: at least one TV display; at least one decoderdriving the display; means for sending a received digital stream havingtime base discontinuities through the decoder for presentation thereofon the display; means for pausing and/or bookmarking the display; meansfor sending a recorded version of the received digital stream throughthe decoder for presentation thereof on the display upon resumption ofplay, the means for sending a recorded version accounting for time basediscontinuities in the stream.
 14. The system of claim 13, wherein themeans for sending a recorded version operates based at least in part onat least one timestamp of at least one frame of the digital stream. 15.The system of claim 14, wherein at least the means for sending areceived digital stream and the means for sending a recorded version areembodied by at least one processor.
 16. The system of claim 15, whereinthe processor is associated with a personal video recorder.
 17. Thesystem of claim 15, wherein the processor uses the timestamp anddiscontinuity count of a last-presented frame in a live video stream toresume play using video data in a storage local to the processor. 18.The system of claim 17, wherein video frames in the storage areassociated with respective timestamps and digital stream discontinuitycounts.
 19. The system of claim 14, wherein if a frame comesunaccompanied by an associated timestamp, a timestamp is calculated forthe frame.
 20. The system of claim 13, comprising a record discontinuitycounter in a recording path and a decode discontinuity counter in adecoding path, and the counters are synchronized with each other.